Problem

The label_referenced_variable_metadata operation was failing with "Column not found in data" error for $label_referenced_variable_role when the variables metadata didn't contain a role field for any variable, as shown in the error below:

This was blocking rule CG0173 which relies on the $label_referenced_variable_role and $label_referenced_variable_name metadata fields.

Root Cause

The operation creates a DataFrame from metadata records returned by _get_variables_metadata_from_standard(). However, when certain fields (like role) are missing from all variables in the metadata, those columns are not created in the DataFrame. This causes a KeyError when rules try to access the expected metadata fields.

For example, if variables metadata looks like:

[
    {'name': 'FATESTCD', 'ordinal': 1, 'label': 'Test Code'},    # No role field
    {'name': 'FATEST', 'ordinal': 2, 'label': 'Test Name'},      # No role field  
]

The resulting DataFrame would not have a role column, causing $label_referenced_variable_role to be missing.

Solution

Modified the LabelReferencedVariableMetadata operation to ensure all expected metadata fields (name, role, ordinal, label) are always present in the metadata records before creating the DataFrame. Missing fields are filled with empty strings.

# Ensure all expected metadata fields are present
expected_fields = ["name", "role", "ordinal", "label"]
for metadata in variables_metadata:
    for field in expected_fields:
        if field not in metadata:
            metadata[field] = ""

Changes Made

1. Updated cdisc_rules_engine/operations/label_referenced_variable_metadata.py:

   • Added logic to ensure all expected metadata fields are always present
   • Missing fields are filled with empty strings to maintain consistency

2. Added comprehensive test test_get_label_referenced_variable_metadata_missing_role_field:

   • Verifies the fix works when role fields are missing from all variables
   • Ensures all expected columns are accessible even when empty

Testing

• ✅ All existing tests pass (202/202 operation tests)
• ✅ New test specifically validates the fix for missing role fields
• ✅ Comprehensive testing of various edge cases
• ✅ Code passes linting checks
• ✅ Backward compatibility preserved

The fix ensures that the $label_referenced_variable_role and $label_referenced_variable_name metadata fields are always available for rule checks, resolving the CG0173 rule blocking issue.

Fixes #1202.

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.